HIRE: Distilling high-order relational knowledge from heterogeneous graph neural networks

Researchers have recently proposed plenty of heterogeneous graph neural networks (HGNNs) due to the ubiquity graphs in both academic and industrial areas. Instead pursuing a more powerful HGNN model, this paper, we are interested devising versatile plug-and-play module, which accounts for distilling relational knowledge from pre-trained HGNNs. To best our knowledge, first propose HI gh-order RE lational ( HIRE ) distillation framework on graphs, can significantly boost prediction performance regardless model architectures Concretely, initially performs first-order node-level distillation, encodes semantics teacher with its logits. Meanwhile, second-order relation-level imitates correlation between node embeddings different types generated by HGNN. Extensive experiments various popular HGNNs models three real-world demonstrate that method obtains consistent considerable enhancement, proving effectiveness generalization ability.